SOCOHEHT BESOBE 

ED 093 971 IH 003 824 



AUTHOfi 
TITLE 

INSTITUTION 
PUB DATE 
NOTE 



S tailings, Jane 

An iBplenentation Study of Seven Follov Through 
Models for Education. 

Stanford Research lnst« , Henlo Fark, Calif. 
Apr 74 

32p. ; Paper presented at the Annual Meeting of the 
American Educational Research Association (5Stb, 
Chicago, Illinois, April 1974) 



EDHS PRICE 
DESCRIPTORS 



IDENTXflERS 



{lF-$0.75 HC-$1.85 PLOS POSTAGE 

^Class Hanageaent; ^Classrooa Observation Techniques; 
Comparative Analysis; Educational Innovation; 
Elementary School Students; Instructional 
Improvement; ♦Models; "^Program Evaluation; Self 
Contained Classrooms; Self Directed Classrooms; 
Teacher Characteristics 
♦Project Follov Through 



ABSTRACT 

This document addresses itself to three aspects of 
the Follov Through Classroom Observation Study: (1) the extent of 
implementation in the classroom, (2) the relationship of training and 
teacher characteristics to classroom implementation, and (3) the 
relationship of studen^w outcome to classroom implementation and 
program characteristics. The sections on sponsor implementation 
provide convincing evidence that teachers are conforming to sponsor 
specifications. There is little deviation betveen classrccms on 
implementation variables, and, except for one sponsor. Educational 
Development Center, the models differe statistically from pon^Follov 
Through. Follov Through teachers have both the understanding of the 
model, since each sponsor's group of teachers described their ovn 
classrooms guite consistently on the structure/flexibility scale, and 
the ability to implement tHie model as proven by the systematic 
observations. The children's test scores are also reported. (BE) 
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AN IMPLEMENTATION STUDY 
OF SEVEN FOLLOVf THROUGH ft^ODEI^ FOR EDUCATION 



The study of implementation has too often been neglected in the 
evaluation of large-*scale social reforms. A problem in evaluating educa- 
tional innovations is that, in too many cases, the programs have not been 
implemented. Evaluation, therefore, could not yield meaningful information 
about either the effectiveness or ineffectiveness of the innovation. 
Charters and Jones (1973, p. 6) state that the collection of test data is 
an "abrogation of professional responsibility" unless evaluators also mea- 
sure, or at least describe, experimental and comparison program differences 
to determine whether the behavior patterns of teachers and students are 
consistent with the planned innovation. The consequence of slighting the 
implementation factor is that "elaborately designed studies may end up as 
appraising non-events, with no one the wiser" (Charters and Jones, p. 5), 

Goodlad's (et al., 1970) Behind the Classroom Door and Weikart's 
(et al., 1973) "Planned Variation from the Perspective of a Model Sponsor" 
also provide information of relevance to the implementation issue, 
Goodlad*s study indicates that although teachers and principals of the 

sampled schools assumed that they had implemented educational innovations, 
findings from observations in the classroom reveal quite the reverse — that 
traditional education practices generally are prevalent: 

"One conclusion stands out clearly: many of the changes we have 
believed to be taking place in schooling have not been getting 
into classrooms ... there seems to be considerable discrepancy 
between teachers* perceptions of their own innovative behavior 
and the perceptions of observers," ((loodlad, et al . , 1970, 
pp. 97 and 98). 

Weikart (et al., 1973) stated that sponsors learned the hard way that 
there was "apparently a vast gulf between the smiles and nods of workshop 
sessions and actual classroom implementation of a model," (p. 12). Despite 
"the enthusiasm of the summer workshops," Weikart found little change in 
the classrooms a few months later. These findings add confirmation to 
Mason's (1973) assertion that evaluation studies frequently find that the 
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innovation "was not actually implemented in the manner specified by the 
developer," 

Although the effectiveness and utility of innovative programs for 
compensatory education har^e come under serious question of late (Jencks 
(1972) and Mosteller and Moynihan (1972)), these studies only evaluated the 
effects of components of (educational systems such as library facilities and 
science laboratories or a<:hieveinent test results. They did not evaluate 
the effects on achievement, of total education^il programs based upon develop- 
mental theories such as those in Follow Through. 

Project Follow Throuf^h was established by the Congress in 1967* when 
it became apparent that a program was needed in the early grades of public 
school that was compatible* with Project Head Start's goals and approaches 
and, therefore, would provide a comparable educational program for econom- 
ically disadvantaged children over a longer period of time. 

Follow Through was originally set up in a "planned variation" research 
design; that is, the goal was to examine the differential effectiveness of 
programs based on divergent educational and developmental theories. The 
program began when a group oi educational researchers, later called Follow 
Through program sponsors, were invited by the government to submit plans 
for establishing their various programs in public schools. This was done 
in order to test their programs' ability to improve the educational achieve- 
ment of economically disadvantaged children. Eleven of the sponsors had 
developed and tried their educational concepts in university settings; 
eight were affiliated witli private research institutes, and three were 
community developed programs. See Egbert (1973) for a history of Follow Through, 

From the inception of the Follow Through evaluation, researchers felt 
that it was important to assess whether the sponsors were effective in 
getting teachers to practice their specified curriculum methods in the 
classroom. In order to obtain information regarding teacher behavior and 
to determine whether a child's day in the classroom corresponded with the 
sponsor's educational prescriptions, it was necessary to observe the 



The legislative authority for Project Follow Through was the Economic 
Opportunity Act of 1964, as amended. 
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classrooms systematically. Thereforei an observation instrument was needed 
because : 

• Systematic observation is a basis for judging the extent to 
which the intended treatment is actually present. 

• Systematic observation is a way to obtain an objective 
description of treatments, 

• Observation can be used as an alternative to more traditional 
methods for assessing child growth and development. 

With the assistance of eight sponsor representatives, the observation 
instrument was developed by SRI in the fall of 1969. The observa- 
tion instrument had to be broad and flexible enough to record the wide 
variety of techniques and approaches used in the various models. As 
examples, Donald BushellVs Behavior Analysis Approach model requires 
systematic recording of the following interactions: the teacher's question, 
the child's response, the immediate feedback to the child, and whether or 
not a token was given, David Weikart's Cognitively Oriented Curriculum 
model needs a way to record the fact that the adults asked open-ended 
questions of the children, and on the basis of these questions the children 
made individual choices and plans for their day's work. The University of 
Pittsburgh's Individualized Early Learning Program needs a way to record 
that the teacher moves about the room providing feedback to individual 
children. The University of Arizona's Tucson Early Education Model and the 
Bank Street College of Education Approach call for a way to indicate that 
the curriculum is interdisciplinary and reflects the child's community. 
The Responsive Educational Program of Far West Laboratory requires a way to 
record that concrete objects are used by the children as they explore their 
environment. For the EDC Open Education program, children have to be 
recorded as independently engaged in a variety of activity centers. 

In order to record the presence of these various components, SRI 
developed an instrument whicrh could (1) assess the physical environment; 
(2) list the classroom activities occurring simultaneously such as social 
studies, reading, math, and art; (3) record who was involved in the activity 
such as teachers, aides, and/or groups of children; and (4) code the verbal 
and nonverbal adult- to- child or child-to-child interaction. 
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Training procedures were developed and observers were trained to observe 
systematically in classrooms all over the country. This effort made it 
possible to assess whether the teacher behavior specified by the sponsors 
actually was occurring in the classrooms. In each of our four years of 
observational research, we found that the classroom teachers and aides 
performed as specified by the sponsor on many of the components that are 
important to the iTiOdel, 

In addition to observing teachers, we also observed individual 
children. Some sponsors had said, "if you want to know if we are imple- 
mented, you must observe our children," For this reason, our system was 
modified to observe individual children. The findings from these child 
observations have been used to evaluate sponsor implementation and to 
measure a child's skill in such behaviors as independence, question asking, 
task persistence, and cooperation. 

The data reported in this paper were collected in the spring of 1973 in 
36 project locations. The sample represents approximately 20 first grade 
and 20 third grade classrooms for each of seven Follow Through sponsors at 
five or more sites per sponsor. Classroom implementation is Judged on the 
basis of two criteria: (1) how uniform are the sponsor classrooms on 
selected implementation variables, and (2) how the sponsor classrooms differ 
from the traditional non-Follow Through classrooms on the same variables. 
The primary purpose of this paper is to present the findings from the 
assessment of the sponsor classroom implementation. In addition, results 
of analyses of the relationships between classroom implementation and 
sponsor's procedures, implementation and teacher characteristics, and 
implementation level and children's test scores are also reported in this 
paper. 



Sponsors of educational models who were observed in Spring 1973: Far West 
Laboratory for Educational Research and Development (5 sites), University 
of Arizona (6 sites). Bank Street College of Education (5 sites). University 
of Oregon (5 sites). University of Kansas (5 sites), High/Scope Educational 
Research Foundation (5 sites), and Education Development Center (5 sites). 
These sponsors were chosen for observation because they met the criterion 
of having five or more sites being implemented. 



A. Methodology Used in the Study of Implementation 

The first step in the assessment of classroom Implementation was to 
describe each educational model in detail. The model descriptions were 
prepared by SRI and reviewed by sponsors and then revised according to the 
sponsor's specifications. The second step was to create variables from the 
codes used on the observation instrument which would describe representative 
elements of each sponsor's model. Each sponsor was sent a variable list and 
asked to rate each variable as to (1) its importance to the model, and (2) 
the expected frequency of occurrence of the variable relative to a conven- 
tional classroom. Thus, a list of variables was selected for each of the 
seven models. These ranged in number from 31 for University of Oregon to 
55 for Bank Street. 

Since the Follow Through programs are intended to be innovative 
programs that represent alternatives to the conventional classroom, a pool 
of non- Follow Through classrooms was used as the standard from which 
Follow Through classrooms should differ in specified ways. The standard 
was established separately for first and third grades. The classrooms of 
each sponsor were assigned an implementation score on each of the variables 
selected for that sponsor. 

Implementation scores for each sponsor were determined by rank ordering 
the non-Follow Through classroom mean scores on each sponsor variable and 
dividing the distribution into five equal parts. There are 35 non-Follow 
Through first grades. The seventh lowest score is the first quintile cut- 
point; the fourteenth lowest score is the second quintile outpoint; the 
twenty-first lowest score is the third quintile outpoint; and the twenty- 
eighth lowest score is the fourth quintile outpoint. Any Follow Through 
classrooms which have a score equal to or below the seventh score is in the 
first quintile. Any Follow Through classroom having a score above the 
twenty-eighth non-Follow Through score is in the fifth quintile. Figure 1 
shows the outpoints for implementation scores for the variable *'Games, Toys, 
Play Equipment Present" for the first grade non-Follow Through classrooms. 

The implementation score for a sponsor will always be a score between 

1 and 5. This represents the position of a Follow Through classroom mean 
relative to the distribution of non-Follow Through means (see Table 1). 
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Figure 1 

GAMES, TOYS, PLAY EQUIPMENT PRESENT 
(First Grade) 



Quintiles: 
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Table 1 



WIDE VARIETY OF ACTIVITIES, OVER ONE DAY 

Implementation Scores 



Sites 



Berkeley 

Duluth 

Lebanon 

Salt Lake City 
Tacoma 



First Grade 
1 2 3 4 5 

4 

3 1 
4 
4 
4 



Third Grade 



1 2 



4 5 



1 
2 



1 3 
1 3 
4 

1 2 
1 1 



Total Classrooms: 
Percent of Classrooms: 



3 17 
15 85 



3 4 13 
15 20 65 



This nonparametric scaling technique was used rather than a technique that 
employs the means and standard deviations of the non-Follow Through class- 
rooms because of the variety of distributions that were encountered in the 
non-Follow Through classrooms. The distributions ranged from the familiar 
bell shape to a j-shaped curve to those with extreme outliers. A parametric 
approach which may be appropriate to one distribution may be inappropriate 
to another. The nonparametric procedures selected for use tend to be less 
sensitive to these differences in distribution than are the more conventional 
parametric procedures. (See Table 2 for sponsor implementation variables 
and quintile outpoints.) 
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For each sponsor's classroom an implementation score was computed for 
each of the sponsor *s variables • A classroom implementation score was 
computed by dividing the sum of the variable implementation scores by the 
highest implementation score possible. The resulting proportion was then 
multiplied by 100 so that it could be expressed in percentage terms. To 
apply the method in an example, the highest possible sum of implementation 
scores for a hypothetical classroom being rated on four variables would be 
4 X 5 - 20, If a classroom had implementation scores of 3, 3, 4, and 5 on 
the individual implementation variables, then the total implementation 
score for the classroom would be x 100 = 75 percent. 

In order to assess the magnitude of the total implementation scores 
for Follow Through classrooms, a total implementation score was also com- 
puted for each non-Follow Through classroom on each sponsor's set of 
implementation variables. The mean and standard deviation of the non- 
Follow Through pooled classrooms are reported fnr each sponsor. Separately 
for first and third grades, one-tailed t tests were computed to test for 
the significance of the differences between each Follow Through sponsors* 
classrooms and the non-Follow Through classrooms, 

B, Results of the Classroom Implementation Study 

Total implementation scores for each classroom for each sponsor are 
presented in Tables 3, 4, 5, 6, 7, 8, and 9, The means and standard 
deviations are presented by grade level based on the scores for all of a 
sponsor *s classrooms in each site and also for pooled non-Follow Through 
classrooms. 

The Far West classrooms in both first and third grades are remarkably 
similar within sites and among sites on total in^lementation scores, with 
the greatest deviation found in the third grade in Duluth (see Table 3), 
Overall, the Far West classrooms at both first and third grades are signif- 
icantly different from non-Pollow Through classrooms whem compared on Far 
West implementation variables. Fifty-one sponsor variables were used in 
this analysis. 

There is a significant difference between the total implementation 
scores in the first and third grade U, of Arizona classrooms and the non- 
Follow Through comparison classrooms (see Table 4) , However, results for 
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Table 3 

TOTAL IMPLEMENTATION SCORES FOR CLASSROOMS BY SITE FOR FAR WEST 



First Grade 

Classroo m Scores/Site Sc ores 

" "SD 

1.5 

2.0 
4.7 
2.1 

3.1 



Third Grade 



Sites 


1 


2 


3 


4 


X 


Berkeley 


(EK)'*' 73% 


76% 


75% 


73% 


74% 


Duluth 


(EK) 78 


80 


80 


78 


79 


Lebanon 


(EK) 80 


76 


80 


78 


78 


Salt Lake 


(EK) 76 


82 


70 


78 


77 


Tacoma 


(EK) 77 


74 


75 


72 


74 


Total Sponsor Scores: 


(N 


= 20) 




76% 


Non-Follow 


Through Scores: 


(N 


= 35) 


65% 



Cla«srooin Scores/Site Scores 



t = 1Q.63 

P< M"' 

EK = children entered school in kindergarten. 



1 


2 


3 


4 


X 


sn 


79% 


71% 


78% 


74% 


75% 


3,5 


76 


64 


77 


73 


73 


6.1 


68 


76 


69 


67 


70 


3.9 


81 


84 


75 


85 


81 


4.5 


74 


77 


74 


72 


74 


2.4 




(N 


= 20) 




75% 


5.4 




(N 


= 36) 




61% 


6.9 




t = 


7.3§ 










P < .001 









Table 4 

TOTAL IMPLEMENTATION SCORES FOR CLASSROOMS BY SITE FOR U OF ARIZONA 

First Grade 



Third Grade 



Classroom Scores/Site Scores Classroom Scores/Site Scores 



Sites 


1 


2 


3 


4 












Des Moines 


(EX)* 77% 


67% 


70% 


•^1% 


Fort Worth 


(El) 78 


76 


73 


73 


LaFayet te 


(El) 75 


62 


80 




Lakewcod 


(EK) 72 


71 


73 


75 


Newark 


(EK) 54 


47 


54 


55 


Lincoln 


(EK) 76 


80 


65 


71 


Total Sponsor Scores: 


(N 


= 23) 




No n- Follow 


Through Scores : 


(N 


= 35) 






t = 


4.00 








p < .001 





X 


SD 


1 


2 


3 


4 


X 
















71% 


4.3 


63% 


55% 


55% 


67% 


60% 


75 


2.6 


64 


7S 


75 


78 


73 


73 


9.5 


60 


64 


73 


75 


68 


73 


1.7 


68 


71 


68 


65 


68 


52 


3.4 


58 


55 


58 


58 


57 


73 


6.5 


72 


71 


72 


77 


73 


69% 


9.2 




(N 


= 24) 




66% 


62% 


5.4 




(N 


= 36) 




61% 








t = 


2.82 












P< 


.01 







EK - children entered school in kindergarten, 
El = children entered school in first grade. 



SD 

6.3 
6.3 
7.3 
2.7 
1.7 
2.7 
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the first grades in LaFayette and Lincoln reveal a greater deviation within 

the first grade total implementation scores than in the deviation of the 
non-Follow Through first grade scores. Also, while the first and third 
grades at Newark show little deviation within the classrooms at the sites, 
their total implementation scores for both grades are lower than those of 
the non- Follow Through classrooms. Thus, while the total implementation 
score for the U, of Arizona classrooms is significantly higher than the 
non-Follow Through classrooms, there is also a greater deviation between 
the total implementation scores of the U, of Arizona classrooms than that 
shown in the non-Follow Through classrooms (based on 48 sponsor variables). 

The Bank Street first and third grade classrooms are notably similar 
in total implementation scores both within and among sites (see Table 5), 
Not only are their implementation scores significantly higher than the non- 
Follow Through scores, but the deviation between Bank Street classrooms is 
also less than the deviation between non-Follow Through classrooms (based 
on 55 sponsor variables)* 

Table 5 

TOTAL IMPLEMENTATION SCORES FOR CLASSRCKDMS BY SITE FOR BANK STREET 

First Grade Third Grade 

Classroom Scores/Site Scores Classroom Scores/Site Scores 



Sites 1 


2 


3 


4 


X 


SD 


1 


2 


3 


4 


X 


SD 


Brattleboro(EK)* 66% 


69% 


62% 




66% 


3.3 


71% 


70% 


67% 




69% 


2.2 


Fall River (EK) 75 


70 


68 


67% 


70 


3.7 


59 


64 


63 


67% 


63 


3.2 


New York (EK) 72 


71 


73 


69 


71 


1.7 


67 


71 


77 


71 


72 


4.1 


Philadelphia 
























(EK)^ 74 


74 


74 


70 


73 


1.7 


64 


68 


63 


68 


66 


2.6 


Tuskegee (El) 76 


75 


72 


76 


75 


1.8 


64 


75 


68 


64 


68 


5.3 


Total Sponsor Scores: 


(N 


= 19) 




71% 


3.7 




(N 


= 19) 




67% 


4.4 


Non-Follow Through Scores; 


(N : 


= 35) 


62% 


5.2 




(N 


= 36) 




62% 


6.9 




t = 


6.53 










t = 


3.15 










P< 


.001 










P< 


.001 









EK = children entered school in kindergarten. 
El = children entered school in first grade. 
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The striking similarity in total implementation scores for both first 
grade and third grade U. of Oregon classrooms is shown in Table 6, There 
is little classroom deviation either within or among sites. The non-Follow 
Through classrooms* total implementation scores are significantly different 
from the U, of Oregon classrooms (using 31 sponsor variables). 

Table 6 

TOTAL IMPLEMENTATION SCORES FOR CLASSROOMS BY SITE FOR U. OF OREGON 

First Grade Third Grade 

Classroom Scores/Site Scores Classroom Scores/Site Scores 



Sites 1 


2 


3 


4 


X 


SD 


1 


2 


3 


4 


X 


SD 


E, St, Louis ^ 
























(EK) 63% 


60% 


59% 


65% 


62% 


2.5 


67% 


61% 


63% 


65% 


64% 


2.7 


New York (EK) 68 


77 


77 




74 


5.4 


63 


66 


61 




63 


2.4 


Racine (EK)^ 62 


61 


61 


63 


62 


1.3 


63 


63 


67 


67 


65 


2.5 


Tupelo (El) 64 


70 


71 


68 


68 


3.1 


68 


62 


70 


60 


65 


4.8 


Providence (EK) 61 


63 


61 


62 


62 


1.3 


61 


67 


59 


55 


61 


5.1 


Total Sponsor Scores: 


(N 


= 19) 




65% 


5.4 




(N 


= 19) 




64% 


3.7 


Non-Follow Through Scores: 


(N : 


- 35) 


60% 


8.7 




(N 


= 36) 




60% 


6.7 




t = 


3.41 










t = 


: 2.41 










p < .001 










P< 


.05 









EK = children entered school in kindergarten. 
El = children entered school in first grade. 



The findings presented in Table 7 reveal few differences in total 
implementation scores either within or among sites for the classrooms of 
the U. of Kansas. VQien total implementation scores were computed for the 
non-Follow Through classrooms on the U. of Kansas variables and compared to 
the U. of Kansas classrooms, significant differences for both grades were 
found (using 34 sponsor variables in this analysis). 

Results for High/Scope^s first and third grades have noticeably 
similar total implementation scores within sites (see Table 8). The 
Greeley site has higher implementation scores than all the other sites in 
both grades. This is an interesting finding because it is the only site 
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Table 7 



TOTAL IMPLEMENTATION SCORES FOR CLASSROOMS BY SITE FOR U. OF KANSAS 



First Grade Third Grade 



Sites 


Classroom 


Scores/Site Scores 


Classroom 


Scores/Site Scores 


1 


2 


3 


4 


X 


SD 


1 2 


3 


4 


X 


SD 


New York (EK)* 


69% 


68% 






68% 


.5 


69% 76% 






73% 


4.7 


Philadelphia 
























(EK) 


74 


74 


72% 


79% 


75 


3,0 


74 73 


71% 


74% 


73 


1,4 


Portageville 
























(EK) 


82 


83 


78 


74 


79 


4.4 


79 71 


71 




74 


4.6 


Kansas City(EK) 


75 


69 


71 


63 


69 


4.9 


79 75 


73 


72 


75 


3.1 


Louisville (EK) 


75 


78 


78 


68 


75 


4.4 


68 75 


68 


72 


71 


3.6 


Total Sponsor Scores: 


(N : 


= 18) 




74% 


5.3 


(N 


= 17) 




73% 


3.3 


Non-Follow Through Scores: 


(N 


= 35) 


62% 


5.7 


(N 


=» 36) 




61% 


7.5 






t = 


7.50 








t = 


6.25 












p < ,001 








P< 


.001 









EK = children entered school in kindergarten. 



Table 8 

TOTAL IMPLEMENTATION SCORES FOR CLASSROOMS BY SITE FOR HIGH/SCOPE 

First Grade Third Grade 

Classroom Scores/Site Scores Classroom Scores/Site Scores 



Sites 1 


2 


3 


4 


X 


SD 


1 


2 


3 


4 


X 


SD 


Greenwood (El) 69% 


67% 


64% 


70% 


67% 


2.8 


64% 


71% 


69% 


69% 


68% 


3.0 


Ft, Walton Beach 
























(El) 74 


68 


73 


72 


72 


2.5 


79 


79 


77 


73 


77 


2.8 


New York City ^ 
























(EK) 64 


66 


66 




65 


1.3 


63 


59 


61 


65 


62 


2.5 


Greeley (EK) 79 


81 


82 




81 


1.8 


78 


77 


84 




80 


3.5 


Denver (EK) 80 


69 


70 


74 


73 


4.7 


68 


64 


74 


74 


70 


4.8 


Total Sponsor Scores: 


(N 


= 18) 




72% 


5.8 




(N 


= 19) 




71% 


7.1 


Non-Follow Through Scores: 


(N 


= 35) 


63% 


5.9 




(N 


36) 




62% 


6.9 




t = 


5.22 










t = 


4.56 










p < .001 










P < 


.001 









EK = children entered school in kindergarten. 
El = children entered school in first grade. 
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with a high percent (25%) of children for whom English is a second language. 
The implementation scores are higher and significantly different at both 
grade levels in comparison to non-Follow Through (based on 47 implementation 
variables). However, the standard deviation between High/Scope site scores 
is similar to that of non-Follow Through classrooms. 

For the most part in this analysis, classrooms are expected to be in 
the upper range of the quintiles when compared to traditional classrooms. 
However, EDC does not expect classrooms to conform to model specifications 
or to differ radically from traditional classrooms • EDC is an "approach" 
to education that recognizes, respects, and incorporates differences into 
its program. Ideas are offered about how to arrange classroom environments 
and how to prepare low cost exploratory materials for children. But by 
their own example of not intruding or insisting upon conformity, the model 
encourages teachers to respect the rights and opinions of children and to 
treat them as individuals. Workshops and guidance are offered by EIDC staff, 
and teacher attendance is voluntary rather than mandatory. Thus, a higher 
rate of variance should be expected among EDC classrooms. 

In EDC's Burlington site, there is remarkably little variation among 
the total implementation scores of classrooms at the site for either 
grade (see Table 9). Philadelphia shows more variation in the total 
implementation scores for the third grade than do other sites. Rosebud *s 
third grade has the least variation of all groups. While the total 
deviation between all classroom implementation scores is not great, the 
EDC classrooms do not differ significantly from the non-Follow Through 
classrooms whose scores were computed on these same 35 EDC variables. This 
lack of difference may indicate that (1) there is no difference between 
EDC classrooms and non-Follow Through classrooms, or (2) the implementation 
variables selected by the sponsor were not sensitive enough to differentiate 
the model classrooms from the non-Follow Through classrooms. 
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Table 9 

TOTAL IMPUESiENTATION SCCXIES FOR CLASSROOMS BY SITE FOR EDC 
First Grade Third Grade 



Classroom Scores/Site Scores Classroom Scores/Site Scores 



Sites 1 


2 


3 


4 


X 


SD 


1 


2 


3 


4 


X 


SD 


Burlington (EK) 64% 


69% 


62% 


66% 


65% 


2.9 


66% 


67% 


66% 


64% 


66% 


1.3 


Philadelphia 
























(EK) 57 


66 


62 


54 


60 


5.6 


71 


65 


70 


56 


66 


7.0 


Paterson (EK) 69 


67 


62 


63 


65 


3.1 


63 


52 


58 


60 


58 


4.6 


Rosebud (EK)^ 59 


53 


52 




55 


3.8 


61 


62 


62 




62 


.9 


Smithfield (El) 66 


61 


72 


62 


65 


5.0 


73 


70 






71 


2.0 


Total Sponsor Scores: 


(N 


= 19) 




62% 


5.6 




(N 


= 17) 




64% 


5.6 


Non-Follow Through Scores: 


(N 


= 35) 


63% 


5.9 




(N 


r 36) 




62% 


8.0 



t = -.36 t = .88 

p< N.S. P< N.S. 

EK = children entered school in kindergarten. 
El = children entered school in first grade. 



B. A Study of the Relationship Between Teacher Characteristics/Training 
and Implementation Scores 

In the study of implementation it is important to try to understand 
what methods or strategies sponsors employed to bring about the changes 
in teacher behavior and what teacher characteristics are related to class- 
room implementation. The evaluation of classroom conformity to sponsor 
goals, which was described in the preceding section by sponsor, leaves no 
doubt that implementation of the Follow Through models has taken place in 
many diverse sites. 

We made an effort to determine (1) which elements in the sponsors' 
inservice teacher training program were effective in the implementation 
process, and (2) which teacher characteristics might be related to success- 
ful implementation. Items from an SRI-developed and -administered Teacher 
Questionnaire regarding the sponsor's teacher training program, teaching 
experience, education, and satisfaction with the sponsor's model were 
analyzed • 

Correlations were computed to examine the relationship between 
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classroom implementation scores and selected teacher characteristics 
and the sponsor's training of teachers. 

Analyses of this d8.ta indicate the following: 

1. Teacher Training Enphasis 

In general I sponsors (other than EDC) seem to provide greater assistance 
to teachers who have lower classroom implementation scores (see Table 10). 

2. Follow Througia Teaching Experience 

Classroom implementation scores and the number of years of teaching 
experience in the Follow Throu^ program are significantly related in the 
Far West and Bank Street models and tend to be positively correlated in 
the case of U. of Oregon and of Kansas, The negative correlations suggest 
that the U. of Arizona and EDC models may be implemented better by teachers 
in their first year of Follow Through experience than by teachers who have 
been with the model for a longer time (see Table 11), 

3. Formal Educaxion 

The data on forms.l education show that teachers with graduate work 
have higher implementation scores in Bank Street and U, of Oregon but not 
in other models (see Table 12) , 

4. Teacher Satisfaction with Model 

Teachers in all the Follow Through models expressed considerable 
satisfaction with their particular model (see Table 13). In only one model 
(U. of Oregon) was there a significant , but negative, relationship between 
implementation and satisfaction (see Table 14), This finding suggests that 
teachers who are best at Implementing the model may be the least satisfied 
with it and may want to change it somewhat. 

5. Structure of Classroom 

Teachers' descriptions of the extent of structure in their classrooms 
is quite distinct (see Figure 2 for items used in this scale) , A low score 
indicates greater structure, while a high score indicates flexibility. (The 
range of scores is from a low of 11 to a high of 55). The teachers' reports 
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Table 15 



FOLLOW THROUGH TEACHER RATINGS 
ON THE CLASSROOMS* STRUCTURE/FLEXIBILITY SCALE* 



Sponsors and Sites 

Far West Laboratory for Kdiicatlonal 
Research and Development 



Uni 



Bank Street College 

0502 Brattleboro, Vt. 

0504 Fall River, Mass. 

050G New York City, P.S. 243K 

0505 Phil. II, Pa. 
0510 Macon Co., Ala. 

Uhiversity of Oregon 

0703 E. St. Louis, 111. 

0707 New York City, P.S. 137K 

0708 Racine, Wise. 
0711 Tupelo, Miss. 
0710 Providence, R.I. 

University of Kansas 

0801 New York City, P.S, 77X 

0803 Phil. VI, Pa. 

0804 New Madrid Co., Mo. 

0806 Kansas City, Mo. 

0807 Louisville, Ky. 

Miph/Scopo Educational Research 
F oundati o n 

0901 Ix^Floro Co., Miss. 

0902 OkaJoosaCo., Fla. 

0903 New York City, P.S. 92M 
090G CIrc-elcy, Colo. 

0907 Denver, Colo. 

Edvication Dovolopmont Center 
ITol Hurlin^'ton, Y^t] 
110:< Plij 1. IV, Pa. 

HOG Paterson, N.Y. 

1107 Ho3oln!d, Texas 

1108 Johnston Co., N»C, 



Teacher N X S.D, 



37 41^ jk9 

2.3 
3.7 
4.0 
6.0 
7.1 

4.0 

4.1 
4.1 

2.3 
2.2 



32 40^ 4^ 

5 41.6 4.8 

8 41.6 2.7 

8 41.3 2.0 

7 34. G 5.0 

4 43.3 1.3 

32 31 .2 4.4 

7 30.1 V 4.8 

5 31.6 5.1 

8 31.0 4.6 

6 31.2 3.4 
6 32.3 5.2 

32 33.6 5.3 

4 35.5 5.2 

6 35.8 2.5 

7 32.4 4.0 

8 32.5 7.7 

7 33.0 5.4 

32 42.9 5.7 

10 39.3 5.1 

5 4G.2 5.2 

4 47.0 2.9 

5 44.0 6.2 

8 42.5 6.2 

37 42^ 4^8 

P. 43 .1 G . 0 

7 38. G 5.3 

7 41.9 3.8 

9 44.8 2.6 

6 45.8 3.3 

60 35.0 4-6 



0201 Berkeley, Calif. 7 40.0 

0204 Duluth, Minn. 7 43.1 

0207 Lebanon, N.H. 8 40.4 

0209 Salt Lake City, Utah 7 42.9 

0213 Tacoma, Wash. 8 40.6 

versity of Arizon a 29 39.8 

0305 Des Moines, Iowa 9 37.1 

0309 Lakewood, N.J. 7 41.1 

0311 Newark, N.J. 6 38.3 

0316 Lincoln, Nebr. 7 43.1 



* Cor r i i c ikMi I for Lhij' sea To was equal to .76. 
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of their classroom practices which conform to the requirements of the 
sponsor's model could not have happened by chance (see Table 15). The 
influence of the sponsors is apparent because (1) there is little deviation 
among the teachers' reports, and (2) the more structured models (U. of Oregon 
and U. of Kansas) are lower on the scale and the more flexible models are 
higher on the scale. 

C. Classroom Instructional Processes and Child Outcomes 

A study of program implementation would have little value if we did 
not believe that classroom instructional processes are related to children's 
cognitive and affective development. 

In one attempt to examine this relationship, correlations v/ere computed 
between classroom implementation scores and means of classroom test scores, 
partialling out the baseline WRAT score. One hundred eight first grade and 
57 third grade classrooms were used in this study. 

The children in the first grade classrooms which had higher implemen- 
tation scores achieved higher scores on MAT reading and arithmetic in the 
Far West, U. of Arizona, Bank Street, and U. of Oregon models (see Table 16). 
Six out of 15 correlations were significant (p < .05). Only EDC had a 
significant negative correlation. The trend in the third grade is also 

toward positive correlations between implementation scores and test scores; 
however, only one out of the 24 correlations were significant (p<.05). 
Third grade children in the better implemented EDC classrooms achieved 
significantly higher scores on the Ravens test. The significant EDC 
negative correlation between implementation scores and first grade math 
scores was reversed in the third grade to a positive correlation. 

Using 166 combined Follow Through and non-Follow Through classrooms, 
partial correlations were also computed for classroom instructional processes 
on the following: selected child behaviors, absence rate, and test scores.* 

In both first and third grades, the tendency is for higher reading and 
math scores to be associated with variables which describe the more struc- 
tured/teacher initiated models. Variables describing the time spent in 



Details of these studies are available at SRI, Menlo Park, California. 
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ro:niiin>j or math activity were also highly correlated wlt!i test scoros . In 
general, a low absence rate, high independence, and high scores on the 
Ravens and Coopersmith tests tend to be associated with the more flexible 
models . 

The lAR Success scale is positively related with variables describing 
the more open classrooms. In general, it seems that children from the 
flexible classrooms take responsibility for their own success but not for 
their frilure. Children from the more highly structured classrooms take 
responsibility for their own failure but attribute their success to their 
teacher's competence or other forces outside themselves. 

Because it is difficult to assimilate and understand all of the findings 
from the correlational studies, some regression models were constructed. 
Variables representing the structured and less-structured models of education 
were selected on an a priori basis. Figure 3 illustrates these models. The 
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Figure 3 
A MODEL FOR EDUCATION 
Instructional Process 



Results 



Initial 
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Motivation 




Opportunity 



Structure and 
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Events 
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Exiting 
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WHAT scores wero UBed i\a baaollno pupil ability and the M/VT Reading; iind 
Math scores were used as outcome criterion measures • Appropriate variables 
were also selected using the Raven's Progressive Matrices for the criterion 
measure. Table 17 presents these findings. One striking aspect of 
Table 17 is the dominance of the Fall 1971 WRAT scores. This score domi- 
nates the variance fcr reading in both the structured and less structured 
models at both grade levels. Instructional variables account for more of 
the variance in the third grade math scores than does the MAT score. The 
baseline score and the instructional variables account for approximately 
the same amount of variance in the Ravens, The total variance explained 
by the structured or less structured models differs very little. The total 
explained variance ranges between a low of 51 percent in the less structured 
math at third grade and a high of 64 percent in the less structured reading 
in the first grade. 

An attempt was made to replicate a regression model suggested by 
William Cooley of the University of Pittsburgh and carried out by John 
Emrick of SRI on the 1972 data. In spite of the fact that a different 
baseline test battery was used in the two studies and the difference in 
time between pretest and posttests, the variance accounted for by entering 

ability (unique) is similar for both studies (see Table 18), However, the 
instructional process (unique) accounts for nearly three times as much of 
the variance in the 1972 study as in the 1973 study. Shared (ability and 
process) is a negative 10 percent for the 1972 study and accounts for only 
one percent of the variance in the 1973 study. The percentages of variance 
accounted for by the separate instructional component variables are presented 
in Table 19, 

The instructional process vai'iable "opportunity" uniquely accounts for 
17 percent of the criterion variance in the 1972 study and for none of the 
variance in the 1973 study. The findings of the large percent of variance 
accounted for by the process variables for the first study are not replicated 
in the second study. Some of the reasons for the differences may be due to 
the fact that the first study was based upon 30 first grades representing 
five sponsors in five southern sites, while the second study had 112 first 
grade classes representing seven sponsors in 25 sites in many geographical 
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Table 17 

PERCENT OF TOTAL CRITERION VARIATION DUE TO GIVEN SOURCE 

Unique 



w 

^ e 
o o 
o 

u u 

o cn 

JD cn 



H 

> 



o 



cn 
o 

W rH 

O cd 



* 
* 



HH Oh > 



c 

•H 

iH cd 

a ^ 



First Grade 



ed 



* 
* 
* 



V 

a 



Reading 

Structure 105 48 10 5 63 37 .01 

Less Structure 105 40 11 13 64 36 .001 

Math 

Structure 105 28 17 9 54 46 .001 

Less Structure 105 21 14 16 51 49 .001 

Third Grade 
Read ing 

Structure 58 38 12 6 56 44 .05 

Less Structure 58 41 10 3 54 46 .05 



Math 

Structure 58 24 36 -2 58 42 

Less Structure 58 21 29 1 51 49 



Ravens 



58 



25 



21 



16 



62 



38 



.001 
.001 

.001 



WRAT score: 

Shared refers to that percent of variance which entering ability 
(Covariables) and the Instructional Process Variables share. 
*** 

p<is the level for test of whether the regression coefficients for the 
instructional variables are all zero. 
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Table 18 

Component Analysis of First Grade Classrooms 



Source of Variance 



Ability (unique) 
Instructional process (unique) 
Shared (ability and process) 



Total Explained 



terror 



Percent of Total Criterion 

Variance Due to Source 

1972 (N = 30) 1973 (N = 112) 



55.7 
25.8 
•10.0 



52.0 
9.0 
1.0 



71.5 



62.0 



28.5 



38.0 



Table 19 

PERCENT OF CHU'TERION VARIANCE UNIQUELY ACCOUNTED FOR 

BY EACH INSTRUCTIONAL PR(X:ESS VARIABLE IN THE COOLEY MODEL 

1972 First Grade 1973 First Grade 
Variables (N = 30) (N = 112) 

Motivation 0% 2% 

Opportunity 17 0 

Structure and Placement 3 0 

Instructional Events 1 2 



locations. We conclude that the findings for the first study are appropriate 
for that particular sample of sponsors, sites, and classrooms but that they 
are not generalizable to other populations. Even so, the regression model 
has utility in organizing complex variables. The overall multiple 
regression coefficients are quite high (about .85 and .79) for the two 
studies . 

In comparing the Cooley model and the Structured and Less Structured 
models using the 1973 first grade reading scores, 4 percent more of the 
variance was explained by entering ability in the Cooley model than was 
explained by entering ability in the Structured or Less Structured models. 
The variance accounted for by the instructional process variables was 
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approximately the same for all three models. Shared variance explains more 
of the variance for the Less Structured model than it does for either the 
Structured or the Cooley model. The total variance explained by the Cooley 
model is 63 percent, by the Structured mod.cl is 63 percent, by the Less 
Structured model is 64 percent. We conclude that the three models are 
equally good in predicting first grade reading scores. 



Summary 

We have addressed three aspects of the Follow Through Classroom 
Observation study in this paper: (1) the extent of implementation in the 
classroom, (2) the relationship of training and teacher characteristics to 
classroom implementation, and (3) the relationship of student outcome to 
clnssroom implementation and program characteristics. 

The sections on sponsor implementation provide convincing evidence 
that teachers are conforming to sponsor specifications. There is little 
deviation between classrooms on implementation variables, and, except for 
EDC, the models differ statistically from non-Follow Through. Most teachers 
in this study serm to differ from those described by John Goodlad (1970) in 
Behind the Classroom Door, for these Follow Through teacher? have both the 
understanding of the model since each sponsor's group of teachers described 
their own classrooms quite consistently on the structure/flexibility scale 
and the ability to implement the model as proven by the systemat c 
observat ions . 

In addition, the better a model has been implemented in a classroom, 
the more likely it is that the children will perform better on the 
criterion tests. Variables associated with the more structured classrooms 
are positively correlated with math and reading scores, whereas variables 
associated with the more flexible classrooms have higher correlations with 
such factors as lower absence rate, independence, and scores on the Ravens 
perceptual problem solving test. 

On the basis of these findings, we conclude that Follow Through planned 
variation is working—not by chance, but by careful design. 
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